

UNITED STATES PATENT APPLICATION FOR 
PREDICTING MANUFACTURING PROCESS FAILURES 



Inventor : 
Evan Randy Kirshenbaum 



CERTIFICATE OF MAILING BY "EXPRESS MAIL" 
UNDER 37 C.F.R. § 1.10 

"Express Mail" mailing label number: ET103507165US 
Date of Mailing: 2-4-2002 

I hereby certify that this correspondence is 
being deposited with the United States Postal 
Service, utilizing the "Express Mail Post Office to 
Addressee" service addressed to Assistant 
Commissioner for Patents, Washington, D.C. 20231 and 
mailed on the above Date of Mailing with the above 
"Express Mail" mailing label number. 



1 H. Horstmann, Reg . tJo . 



Paul H. Horstmann, Reg.^fiTo. 36,167 
Signature Date: 1 - H - ^ 



BACKGROUND OF THE INVENTION 



Field of Invention 

The present invention pertains to the field of 
manufacturing processes. More particularly, this 
invention relates to predicting manufacturing process 
failures . 

Art Background 

A typical manufacturing process includes an 
arrangement of process steps. Typically, each 
process step performs one or more functions 
pertaining to a product undergoing manufacture. In 
addition, a typical manufacturing process usually 
includes an end of line process step at which 
finished products are tested for failures. Tests may 
also be performed at many points along the process. 

It would be desirable to predict which products 
undergoing manufacture will be fail and render the 
prediction at process steps earlier in the 
manufacturing process than the end of line tests. 
Such a prediction may be used to save the costs of 
running potentially defective items through to the 
end of the manufacturing process as well as the cost 
of manufacture for good parts that are later 
irrecoverably attached to defective parts. Such cost 
savings may reduce the overall cost of manufacturing 
products . 
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SUMMARY OF THE INVENTION 



A method for generating a predictor of failure 
of a manufacturing process is disclosed. The 
5 predictor is generated by generating a population of 
candidate solutions for the predictor and then 
determining the fitness of each candidate solution 
using a fitness case pertaining to the manufacturing 
process and a set of costs associated with the 
10 manufacturing process. The fitness may then be used 
to formulate a next generation of candidate solutions 

r*" 

CJ for a design method modeled on evolution. 

CI 
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S| Other features and advantages of the present 

yjj 15 invention will be apparent from the detailed 
Ul description that follows. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



The present invention is described with respect 
to particular exemplary embodiments thereof and 
reference is accordingly made to the drawings in 
which: 

Figure 1 shows an example manufacturing process 
that includes a predictor generated according to the 
present techniques ; 

Figure 2 shows steps in a method for generating 
a predictor of failure of a manufacturing process; 

Figure 3 shows a fitness evaluator which 
determines a fitness indication for a candidate 
solution for a predictor; 

Figure 4 shows a method for determining a 
fitness of a candidate solution. 
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DETAILED DESCRIPTION 



Figure 1 shows an example manufacturing system 
100 that includes a predictor 60 according to the 
present techniques. The manufacturing system 100 
includes a set of process steps 10-24 which are 
arranged along a set of manufacturing lines 40-48. 
Each process step 10-24 performs one or more 
functions associated with the manufacture of a 
product 92 . 

Examples of functions that may be performed at 
the process steps 10-24 include test and/or 
measurement functions, mechanical processes including 
machining and assembly, chemical processes, and 
electrical processes to name just a few examples. 
The selection and arrangement of the process steps 
10-24 is designed in accordance with the nature of 
the product 92 . The example arrangement the process 
steps 10-24 is provided only for purposes of 
illustration. The present teachings are readily 
applicable to any manufacturing process producing any 
product using any selection and arrangement of 
process steps . 

In the example manufacturing system 100, a first 
sub-assembly is manufactured along the manufacturing 
line 40 and a second sub-assembly is manufactured 
along the manufacturing line 42. The first and 
second sub-assemblies are assembled together at the 
process step 18 into a final sub-assembly. An 
example result shown at the process step 18 is a 
final sub- assembly 90 . The final sub -assemblies are 
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merged with product housings carried on the 
manufacturing line 46 at the process step 20. The 
manufacturing line 48 takes the final sub-assemblies 
with product housing to the end of line 2 4 where 
5 final verification on manufactured product is 
performed - for example the product 92 . 

The predictor 60 generates a prediction 80 of 
eventual success/failure in the manufacture of a 
10 final product that will include a final sub-assembly 
currently located at the process step 18. In example 
shown, the predictor 60 generates the prediction 80 
for the final sub-assembly 90 that will be included 
Q\ in the product 92. The prediction 80 if indicating 

yi 15 failure may be used by the manufacturing system 100 

«fi to discard the final sub-assembly 90 or send it back 

11] 

for rework, etc. This bypasses one or more of the 
process steps 10-24. 
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J a 20 The predictor 60 may be embodied as a 

W 

f|j classifier. Possible implementations of a classifier 

are numerous and include computer programs, Bayesian 
networks, neural networks, etc. 

25 The predictor 60 determines the prediction 80 

using a set of process data 78 obtained at the 
process step 18 and sets of process data 70-76 
obtained at the prior process steps 10-16. Some or 
all of the process data 70-78 may pertain to the 

30 final sub-assembly 90 In addition, some or all of 
the process data 70-78 may pertain to parts or sub- 
assemblies other than those included in the final 
sub-assembly 90. 
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For example, the process data 78 may include 
measurements performed on the final sub-assembly 90 
obtained at the process step 18 and/or test results 
from tests performed on the final sub-assembly 90 at 
the process step 18. In another example, the process 
data 74 may include measurements performed on a sub- 
assembly included in the final sub-assembly 90 and 
obtained at the process step 12 and/or test results 
from tests performed on the sub-assembly included in 
the final sub-assembly 90 at the process step 12. In 
yet another example, the process data 72 may include 
measurements performed on a set of parts that include 
a part which is included in the final sub-assembly 90 
and which were obtained at the process step 14 . 

A predictor according to the present teachings 
may be associated with any process step in a 
manufacturing process. For example, predictors 
according to the present techniques may be provided 
at process steps along the manufacturing lines 40-42 
to indicate the likelihood of success/failure of 
final products that include instances of the first 
and second sub-assemblies and/or parts thereof. In 
addition, predictors according to the present 
techniques may be provided anywhere along the 
manufacturing line 48. 

The example predictor 60 is adapted to the 
manufacturing system 100 and the process step 18. In 
one embodiment, the predictor 60 is adapted by 
evolving the predictor 60 using a set of fitness 
cases which are obtained from historical records of 
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the process data 70-78 and the corresponding 
success/failures of the resulting products. 

Figure 2 shows a method for generating a 
predictor of failure of a manufacturing process. The 
method shown may be used to generate the predictor 6 0 
or to generate predictors for other portions of the 
manufacturing system 100. 

At step 110, a candidate solution for the 
predictor 60 is generated. An example of a candidate 
solution is a computer program. Another example of a 
candidate solution is a Bayesian network. Yet 
another example of a candidate solution is neural 
network. These are only example implementations of 
candidate solutions and others may be used in 
accordance with the present teachings. 

A candidate solution in terms of an evolutionary 
design methodology may be referred to as an organism. 
The candidate solution generated at step 110 may be 
an organism in an initial population of organisms of 
an evolutionary methodology or may be an organism in 
a subsequent population of organisms in the 
evolutionary methodology. Organisms in an initial 
population may be generated randomly and organisms in 
subsequent populations may be formed by selecting 
parent organisms from a previous population based on 
their fitness and combining the parent organisms 
using operators which are modeled on the genetic 
processes of mutation and/or cross-over. 
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At step 112, a fitness of the candidate solution 
obtained at step 110 is determined using a set of the 
fitness cases pertaining to the manufacturing process 
and a set of costs associated with the manufacturing 
system 100. The fitness cases may be obtained from 
historical data pertaining to prior products yielded 
by the manufacturing system 100. The fitness of the 
candidate solution may be reinforced using cost data 
associated with the costs of mis-predictions by the 
candidate solution . 

Figure 3 shows a fitness evaluator 14 0 which 
determines a fitness indication 146 for a candidate 
solution 150 for the predictor 60. The fitness 
evaluator 140 obtains records from a process data 
history 142. The process data history 142 includes 
records of the process data 70-78 that was obtained 
for a set of prior final sub-assemblies yielded at 
the process step 18 along with indicators of the 
success/failure of the corresponding products 
determined at the end of line 24. 

In addition, the fitness evaluator 140 obtains a 
set of manufacturing cost data 144 pertaining to the 
manufacturing system 100. The manufacturing cost 
data 144 indicates the costs associated with running 
the manufacturing system 100 and its sub-elements as 
well as revenues that may be obtained from the sale 
of final produces. The manufacturing cost data 144 
may also include information such as the cost of 
restocking parts, etc. 
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Figure 4 shows a method for determining a 
fitness of a candidate solution. The fitness 
evaluator 14 0 in one embodiment performs the steps 
shown when determining, the fitness indication 14 6 in 
response to the candidate solution 150. 

At step 120, the fitness evaluator 140 obtains a 
prediction 154 from the candidate solution 150 in 
response to a fitness case 152 obtained from the 
process data history 142. An example of a fitness 
case is a record of the process data 70-78 obtained 
for a prior final sub-assembly yielded at the process 
step 18. The fitness evaluator 140 obtains the 
prediction 154 by applying the fitness case 152 to 
the candidate solution 150 which may be implemented 
for example as a classifier. The resulting 
prediction 154 indicates the likelihood of 
success/failure of a final sub-assembly which is 
characterized by the fitness case 152. 

The prediction 154 may be a number that 
indicates confidence of success/failure. The 
prediction 154 may be compared to a threshold value 
to yield a boolean success/failure indication. 
Alternatively, the prediction 154 may be treated as a 
probability which is used to scale the reinforcements 
applied to the candidate solution 150. 

At step 122, the fitness evaluator 140 compares 
the prediction 154 to an actual success/failure 
result associated with the fitness case 152. The 
following matrix illustrates the possible 
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relationships between the prediction 154 and the 
actual result associated with the fitness case 152. 





Success 
Predicted 


Failure 
Predicted 


Actual Success 


True Negative 


False Positive 


Actual Failure 


False Negative 


True Positive 



At step 124, the fitness evaluator 140 
reinforces the fitness of the candidate solution 150 
using the manufacturing cost data 144 if the 
prediction 154 corresponds to the actual result. 
Reward reinforcements correspond to true positive 
and/or true negative relationships. 

Conversely, at step 126 the fitness evaluator 
140 reinforces the fitness of the candidate solution 
150 using the _ manufacturing cost data 144 if the 
prediction 154 does not correspond to the actual 
result. Reinforcement penalties correspond to false 
positive and/or false negative relationships. 

In one embodiment, the reinforcement adjustment 
for a true positive result from the candidate 
solution 150 is the cost saving yielded by not 
running a final sub-assembly from the process step 18 
to the end of line 24 as indicated in the 
manufacturing cost data 144. In such an embodiment, 
the reinforcement penalty for a false positive result 
from the candidate solution 150 is the loss of 
revenue from selling a final product minus the cost 
saving from not running a final sub-assembly from the 
process step 18 to the end of line 24. There is no 
reinforcement, neither reward nor penalty, for 
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negatives whether true or false. This embodiment 
involves measuring the cost/benefit of adding a test 
to a running line. For negatives there is no effect 
because the product goes on as usual. For positives, 
on the other hand, cost is saved and in the case of 
false positives revenue is lost. 

In another embodiment, the reinforcement 
adjustment for a true negative result from the 
candidate solution 150 is the revenue obtained from 
selling a final product minus the cost of running a 
final sub-assembly from the process step 18 to the 
end of line 24. In such an embodiment, the 
reinforcement penalty for a false negative result 
from the candidate solution 150 is the cost of 
running a final sub-assembly from the process step 18 
to the end of line 24. There is no reinforcement 
adjustment for positives whether true or false. This 
embodiment involves treating the cost of reaching a 
point in the process and sunk and evaluating the 
expected return on the rest of the line. For 
positives, the product is discarded so there is no 
further cost or return. For negatives, the cost of 
reaching the end of the line is incurred and in the 
case of true negatives the revenue is obtained. 

In yet another embodiment, the reinforcement 
penalty for a false negative result from the 
candidate solution 150 is the cost of running a final 
sub-assembly from the process step 18 to the end of 
line 24 while the reinforcement penalty for a false 
positive result is the cost needed to manufacture 
another final sub-assembly to be used as a 
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replacement up to and including the process step 18. 
This embodiment involves measuring the distance from 
what is deemed a "perfect predictor" and examining 
only the cost incurred by being wrong. 

The reinforcements based on costs may be used as 
the fitness indication 146 expressed in terms of 
dollars. Multiple fitness cases may be applied to 
the candidate solution 150 and the corresponding 
fitness indications 146 may be used to calculate 
overall average fitness value for the candidate 
solution. 

In some embodiments, the fitness of a candidate 
solution may not take into account costs. For 
example, the fitness may be based on the accuracy of 
the candidate solution - e.g. the number of true 
positives and true negatives divided by the number of 
fitness cases. Other measures that may be employed 
include recall, precision, the geometric mean of 
recall and precision, as well as a variety of others. 

The fitness values determined for different 
candidate solutions may be used to select the 
25 candidate solutions to be parents for generating a 

next generation of candidate solutions. For example, 
a set of candidate solutions having higher fitness 
may be more likely to be selected as parents than the 
candidate solutions having lower fitness. The 
3 0 candidate solutions in the next generation may have 
one or more characteristics obtained from their 
corresponding parents. 
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Some final sub-assemblies which the predictor 60 
classifies as likely failures may be allowed to 
complete the manufacturing process. The predicted 
failures may then be tested at the end of line 24 to 
verify the efficacy of the predictor 60. 

Some final sub-assemblies which the predictor 60 
classifies as likely failures and which are allowed 
to complete manufacture may be used in combination 
with some of the final sub-assemblies which the 
predictor 60 classifies as likely successes to obtain 
data for additional fitness cases that may be used to 
evolve an improvement to the predictor 60. In 
addition, some final sub-assemblies which the 
predictor 60 classifies as likely failures may be 
allowed to complete manufacture to verify the 
efficacy of the predictor 60 even if it is not to be 
improved, for example, by retraining. 

The present teaching may be employed to evolve 
predictors at each step of a manufacturing process. 
The fitness of predictors may depend on the selection 
of earlier or later predictors because the available 
precess data as well as cost increase along the 
manufacturing process. For example, a later 
predictor may take into account only those fitness 
cases that would be passed, i.e. corresponding 
products not discarded, by earlier predictors. For 
negative predictions a predictor may take into 
account only the cost of reaching a later predictor 
would indicate a positive result. 
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A predictor according to the present teachings 
may use data collected from prior sub-assemblies. 
This data may be in raw form or aggregate form such 
as mean, range, histogram, etc. For example, a 
particular combination of values may not in of 
themselves be reason to declare a part bad, but if a 
relatively large percentage of prior sub-assemblies 
have that combination then it increases the 
confidence in a failure prediction. A special case 
of this may involve examining the most recent 
pass/fail decisions from the predictor. In addition, 
data may be available from prior parts at subsequent 
steps in the process even up to a final test. In 
general, data collected temporally prior to the 
decision being made may be employed by a predictor. 

The foregoing detailed description of the 
present invention is provided for the purposes of 
illustration and is not intended to be exhaustive or 
to limit the invention to the precise embodiment 
disclosed. Accordingly, the scope of the present 
invention is defined by the appended claims. 
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